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Abstract — For MIMO two-way relay channel, this paper pro- 
poses a novel scheme, VBLAST-PNC, to transform the two 
superimposed packets received by the relay to their network 
coding form. Different from traditional schemes, which tries to 
detect each packet before network coding them, VBLAST-PNC 
detects the summation of the two packets before network coding. 
In particular, after firstly detecting the second layer signal in 
2-by-2 MIMO system with VBLAST, we only cancel part of 
the detected signal, rather than canceling all the components, 
from the first layer. Then we directly map the obtained signal, 
summation of the first layer and the second layer, to their network 
coding form. With such partial interference cancellation, the 
error propagation effect is mitigated and the performance is thus 
improved as shown in simulations. 

Index Terms — Multiple Input Multiple Output, Physical layer 
Network Coding, Two Way Relay Channel, VBLAST. 



I. Introduction 

IN wireless networks, the use of relay is attracting increas- 
ing attention [1,2] because of its many advantages. Among 
the relay channels, two-way relay channel (TWRC, as shown 
in Fig. 1) is a especially interested due to the almost double 
spectral efficiency with the physical layer network coding 
(PNC) [3] transmission scheme. It was further proved in [4, 5] 
that PNC can approach the capacity of TWRC in high SNR 
region. 

Another spectral efficiency boosting technique is Multiple 
Input and Multiple Output (MIMO), which has been widely 
used in wireless systems. Therefore, it is of great interest to 
combine PNC and MIMO to further improve the wireless 
spectral efficiency. A straightforward way is to divide the 
MIMO transmission into parallel SISO streams by precoding, 
so that PNC can be implemented on each steam [6], However, 
the precoding requires not only transmitter side channel state 
information (CSIT) but also strict time and carrier phase 
synchronization between the two end nodes. 

MIMO NC scheme [7] is more practical, where only re- 
ceiver side CSI (CSIR) is needed. In MIMO NC, the relay 
node detects each end node's packet with traditional MIMO 
detection and then combines them with network coding. These 
schemes failed to exploit the fact that the relay does not 
need each end node's individual information. Hence, the 
performance is limited by over-detection. 

In [8], we have proposed a novel MIMO PNC scheme 
based on linear detection, which will be referred as linear 
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MIMO PNC in this paper. Linear MIMO PNC tries to detect 
the summation and the difference of the two end node's 
packets before transforming them to the network coding form. 
With similar complexity and CSIR requirement, it significantly 
outperforms MIMO NC. However, due to the performance 
limit of linear detection, linear MIMO PNC's performance is 
poor under bad channel conditions. 

Besides the linear detection, another popular MIMO de- 
tection method is VBLAST (Vertical Bell Laboratories Lay- 
ered Space-time) which can achieve much better performance 
with an acceptable increase in complexity. The VBLAST 
architecture was first proposed in [9] where a code block is 
de-multiplexed into different layers and each is transmitted 
through a particular antenna. At the receiver, these layers 
are successively detected, where the detected interference are 
canceled and the unknown interferences are nulled by linearly 
weighting the residual signal vector with a ZF (zero-forcing) 
nuU vector (ZF VBLAST). A low complexity ZF VBLAST 
scheme is proposed in [10], where the channel matrix is 
rewritten in terms of the QR decomposition as H = QR. 
The inverse of unitary matrix Q was then multiplied to the 
received signal before estimating the transmit information. In 
order to find the optimal detection order, [10] further proposed 
the sorted QR decomposition algorithm, ZF-SQRD. 

In this paper, we combine the basic idea of PNC and the 
QR VBLAST MIMO detection scheme and propose VBLAST 
PNC. Our scheme only requires receiver side channel state 
information and symbol level synchronization between the end 
nodes, as in general virtual MIMO system. The basic idea 
of VBLAST PNC in a 2-by-2 MIMO system is as follows. 
With QR decomposition, the relay first detects the second 
layer (one end node) signal. Rather than canceling all the 
component of the second layer signal from the first layer as 
in traditional VBLAST detection, we only subtract a part of 
the second layer information and directly map the residual 
signal (including both second and first layer information) 
to the network coding form. With such partial interference 
cancellation, the error propagated from the incorrectly detected 
second layer is significantly decreased. Thus, the system 
performance is improved. We then extend our VBLAST PNC 
to a detection scheme with an optimal order as in ZF-SQRD, 
and even better performance is achieved. Numerical simulation 
is done to compare the performance of VBLAST PNC with 
linear MIMO PNC and MIMO NC schemes. The results show 
that VBLAST PNC can achieve much better BER performance 
than linear MIMO PNC and VBLAST MIMO NC. 



SUBMITTED TO IEEE WIRELESS COMMUNICATIONS LETTES 



2 



where A is a small quantity. Then eq.(l) can be rewritten 



Fig. 1: Two-Way Relay Channel 



II. System Model and Illustrating Example 
A. System Model 

This paper is mainly based on the TWRC in Fig. 1, where 
the relay is equipped with 2 antennas and the each end node 
is equipped with single antenna. 

The transmission consists of two phases. In the uplink 
phase, both the end nodes transmit their packets to the relay 
node simultaneously using QPSK modulation Q. We assume 
that the two packets arrive at the relay node in a symbol level 
synchronization. In that way, the superimposed signal received 
by the relay is: 



2/1 

2/2 



hxxxx 
h 2 ixi 



h\ 2 X 2 
h 22 X 2 



■ ni 
n 2 



(1) 



where t/j denotes the received signal at the i-th antenna of 
the relay node; h{ t j is zero mean complex Gaussian random 
variable, which denotes the channel coefficient from the end 
node Nj to the z-th antenna of the relay node; Xi € {±1 ± 
\J— l}is the transmitted signal with QPSK modulation of the 
end nodeNi, and m demotes the complex Gaussian noise with 
zero mean and a 2 variance for each dimension. 

In this phase, the full channel information is available at the 
relay node. Rewriting the received signal in the vector form 
as 

Y = HX + N (2) 

Then, the relay node tries to extract some useful information 
from Y and transforming it to the network coded form x\ (Bx 2 . 
The detailed processing will be illustrated later, which is also 
the focus of our paper. 

In the downlink phase, the relay broadcasts the network 
coded packet to both end nodes. After receiving the packet 
from the relay, the end nodes extract their target packets with 
the help of their own information. 

B. Illustrating Example 

Now, we present an example to illustrate the basic idea of 
VBLAST PNC and its superiority over VBLAST NC, with a 
simple channel realization. 

In the first phase, the transmission in (2) can be regarded as 
a 2 x 2 MIMO system (two transmit antennas and two received 
antennas). The relay node's goal is to acquire an estimate of 
xi @x 2 . In the traditional MIMO NC scheme, the relay node 
decodes x\ and x 2 explicitly before network encoding them. 
Nevertheless, this scheme is suboptimal. 

Consider an ill-conditioned channel matrix H 



H 



1 1 + A 
A 



(3) 



as 
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x 1 + (1 + A)x 2 
Ax 2 + n 2 



(4) 



With QR ZF VBLAST detection, x 2 is first detected as 



x 2 = -ry 2 =x 2 + ^n 2 (5) 

According to the processing of VBLAST, we cancel the 
interference x 2 in y\ and obtain the estimate of x\ as 

Si = 2/1 - (1 + A)5^ 

= xi + (1 + A)x 2 + m - (1 + A)(x 2 + ±n 2 ) (6) 



x% + ni 



l+A 



n 2 



From (O and (|6), we can find that the SNR of x\ tends to 
zero when A is very small, it means that the VBLAST NC 
scheme can't accurately estimate the target signal X\ © x 2 in 
this case. 

Based on the basic idea of PNC, the relay can estimate the 
target signal x\ © x 2 from x\ + x 2 . Therefore, we can first 
estimate x\ +x 2 rather than estimate individual information of 
xi, x 2 , from Y, The particular processing is as follows. After 
estimating x 2 , we subtract Ax^ rather than (1 + A)x^ from 
y%. Then, we can obtain 



y' 1 =y l - Ax 2 =xi +x 2 +ni-n 2 



(7) 



With the PNC mapping [3], we can directly map y[ in (7) 
to the target information xi © x 2 . Since the noise in (7) is 
small (independent of A), the performance of this scheme is 
much better than the MIMO NC scheme. 

This example indicates that MIMO PNC may significantly 
outperform the traditional MIMO NC. In the following sec- 
tions, we introduce the proposed VBLAST PNC scheme in 
detail. 

III. VBLAST PNC Scheme 

In this section, we first briefly review the MIMO NC scheme 
based on QR VBLAST detection algorithm [9, 10] for a 
comparison. After that, we elaborate the proposed VBLAST 
PNC. 

A. MIMO NC based on QR VBLAST 

Consider the 2-by-2 MIMO system in (2). The channel 
matrix can be decomposed with QR decomposition so that 
H =QR, where the matrix Q is a unitary matrix (orthogonal 
columns with unit norm) and R is an upper triangular matrix. 
Multiplying the received signal Y by Q H , we can obtain the 
calculated signal 

W = Q H Y = RX + U (8) 

where (-) H denotes the matrix conjugate transpose and the 
new noise vector U — Q H N has the same distribution as N. 
The scalar form of W is 



W 2 



ruxi 
r 22 x 2 



r\ 2 x 2 

"2 



(9) 



Our scheme can be easily extended to more general modulations. 



Owing to the upper triangular characteristic of R, we can 
easily estimate the second layer signal (x 2 here) as: 
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(10) 



Since the QPSK modulation is adopted, the above signal can 
be demodulated with hard decision as 



xi = sign(x 2 ) 



(11) 



Note that the hard decision in (fTTT) is performed for the real 
part and the imaginary part respectively. According to the basic 
idea of VBLAST, after the second layer signal X2 is detected, 
we can then detect the first layer signal (xi here) by canceling 
the interference of X2 as: 



x\ = sign(xi) — sign 



wi - n 2 x 2 



(12) 



After obtaining the individual decisions of x\ and X2, a 
straightforward way to calculating x\ © X2 is to combine the 
estimates of x\ and X2 (obtained in (1 1) and (12) respectively): 



X\ © x 2 = Xi © x 2 



(13) 



Hereafter, we refer to this scheme as VBLAST NC. VBLAST 
NC detects the signals separately with VBLAST algorithm, 
and then encodes them into the network-coded form. However, 
it may perform poorly as shown in our illustrating example. 

As in the VBLAST processing above, exchange of the 
detection order between xl and x2 (first detect x\ and then 
cancel it before detecting X2) is also workable. As shown in 
[10], the detection sequence is crucial to the performance of 
VBLAST because of the risk of error propagation. To obtain 
a better performance, we can permute the columns of the 
channel matrix H before the QR decomposition. By carefully 
selecting the permutation pattern, the post-permuted R has a 
larger r 22 . As a result, the SNR of the second layer is increased 
and the BER is decreased, and the error propagation effect is 
also decreased. We can also adopt this sorted algorithm [10] 
in our VBLAST PNC, as elaborated in the following part. 

B. VBLAST PNC Scheme 

In VBLAST, the diversity order of the first layer should be 
larger than the second layer in theory under the assumption 
of clean interference cancellation. In practice, however, the 
diversity order of both layers is the same [11]. The main reason 
is the error propagated from the erroneously detected second 
layer signal. To mitigate the error propagation and better the 
performance, we propose VBLAST PNC scheme which only 
cancels a part of the detected second layer information. After 
the cancellation, we require the remaining signal to be in the 
form of xi + kx2 + u\ where k is an integer. We can then 
directly map this signal to the target signal ^iffiiz^ by applying 
PNC mapping, without explicitly detecting x\. Since the PNC 
mapping has similar performances as the ordinary point-to- 
point transmission, our VBLAST PNC could achieve a better 
performance by mitigating the error propagation effects. The 
detail of VBLAST PNC is as follows. 

We rewrite W\ in (9) as 



where k is an integer to be determined later. In ([TJ we regard 
xi+kx 2 as the signal to be estimated and (ri 2 -krn)x 2 as 
interference to be cancelled. 

In order to decrease the effect of error propagation, we 
must minimize (r 12 -A:r 11 ). For example, if k = r i2 /rn, no 
interference needs to be cancelled and there will never exist 
error propagation during the detection of the signal in < fT~4b . 
Taking the integer requirement of k into account, we can 
determine the value of k as 



k = \real(r 12 )/rii\ 



(15) 



where \t\ means the integer nearest to t. In (15), r\2 is a 
complex variable and m is a real variable, and we only take 
account the real part to calculate k in this paper. 

After cancelling the interference with the hard estimate of 
X2 in (11), we can obtain the estimate of x\ + kx2 as 

(ri2 - krn)x2~ 



Xi + kx 2 = 



Wi 



(16) 



Finally, the relay node estimates x\ © X2 only from X\ + 
kx 2 as long as k does not equal to 0. When k>0, intuitively, 
abs(xi + kx 2 ) is larger for X\ © X2 = 0, while it is smaller 
for xi © x 2 = 1. When k<0, the situation reverse. Then, 
for each dimension (real part or imaginary part) signal, the 
corresponding decision rule is 

Xi © x 2 = sign(k)sign(abs(xi + kx 2 ) — 7) (17) 

where 7 is the decision threshold and its optimal value can 
be calculated as in [3]. In high SNR, we can simplify the 
calculation of 7 and set it to abs(k). Then we have 



Xi © x 2 



sign(abs(xi + kx 2 ) — k) when k > 
sign(—k — abs(xi + fca^)) when k < 



U'l 



r n (xi + kx 2 ) + (ru - kr n )x 2 + u x (14) 



(18) 

To further improve the performance, we can extend our 
scheme to the sorted VBLAST PNC, where the optimal de- 
tection order is chosen as the tradition VBLAST. In particular, 
we exchange the columns of H to obtain a larger r 22 . Then, 
the VBLAST algorithm is performed on the new H. 

IV. Numerical Simulation 

In this section, we present the simulation results for the 
proposed VBLAST PNC. To compare its performance, we also 
show the simulation performance of linear MIMO PNC, the 
performance of linear MIMO NC and VBLAST NC. 

The simulation setting is mainly based on the system model 
in section II. In particular, we use QPSK modulation and set 
the packet length to 10 6 . The wireless channels are assumed 
to be block fading with each entry of the channel matrix to be 
independently complex Gaussian distributed over N(0, 1). The 
noise is Gaussian distribution with N(0, a 2 ) and the SNR of 
the system is defined as 1/er 2 . Simulation results are measured 
in terms of bit error rate (BER) of xi © X2 at the relay node 
since the broadcast phase is straightforward. 

In Figure 2, we plot the BER performance of different 
schemes. As shown in the figure, the proposed MIMO PNC 
schemes always outperform their counterparts. Specifically, 
our VBLAST PNC outperforms VBLAST NC by about 0.5 
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Fig. 2: BER performance of MIMO PNC and MIMO NC 
schemes 



dB at a BER of 1CT 3 ; the sorted VBLAST PNC outperforms 
sorted VBLAST NC by about ldB. In sorted VBLAST, the 
more performance improvement mainly comes from the fact 
that the average value of ru is smaller and the interference 
to be cancelled, (r 12 — krn)x2, is smaller. 

V. Conclusion 

In this paper, a novel signal detection and network encoding 
scheme, VBLAST PNC, is proposed to extract x% © Xi at 
the relay node in MIMO TWRC. The basic idea is that the 
relay node first uses partial interference cancellation to obtain 
xi+kx 2 during VBLAST detection process and then converts 
it to x\ © x-i with PNC mapping. With partial interference 
cancellation, error propagation effect is mitigated and the 
performance is significantly improved. The simulation results 
verified the performance advantages of VBLAST PNC under 
the setting of random Rayleigh fading channel coefficients. 
Our scheme is of great interest in practice since only CSIR, 
symbol lever synchronization and low complexity are needed. 
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